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ABSTRACT 

This technical report documents the development and 
initial empirical testing of a practical method for estimating the 
capacity of Air Force operational units to conduct on-xhe-job 
^"training (OJT) . The method is based on a conceptual model of OJT 
capacity that describes the complex interrelationships among the 
level and guality of mission performance achieved, and the amount and 
quality of training provided, by an operational unit. The unit f s 
capacity to conduct OJT is then determined as the maximum amount of 
training sustainable by the unit without compromising established 
training guality and mission performance standards. The model has 
been empirically tested by estimating its basic structural equations, 
to the extent possible, using existing data for six operational 
units. Despite substantial limitations in data availability, the 
empirical analyses reveal, for all units studied, a persistent and 
strong inverse relationship between the training load cf a unit and 
the mission performance quality of the unit. These favoraoie results 
indicate that further development of the OJT capacity estimation 
methodology can reasonably be expected to significantly enhance Air 
Force training management capabilities. (Author) 
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iNTRODUCTI' 



A detailed ana ! yai .> ': r;: u ~ r.- t he- '■ * h \ vai nil* - (■■~V7) 

-v.; tern has explicit iy • ...a need fcr bet cor and more oaal i 

applied not hods of :;:onii; '.: Ln.- . ::ri c va luat in. tna Air Force O.JT r re >\rar, 
in terns of costs, quality . :' feet i venose training provided.. . . r; 

Moreover, the Air Force ir. t.r acera: , in recently ecv.ip le zee 
F- -rational Mriiura-cnt 1 u.v; c r t : an of the Air Force OJT program > has 
concluded that lack i-£ coat arc capacity information is a ma-jo^ hind- 
rance to the efficient management of the program. Yet, estimation of 
tr~ ampacity of Air Force operational units to conduct OJT has receive 
Ivll]^ attention prior to this report. Therefore, the. fundamental 
objective of this study is to develop and demonstrate a practical 
ip.-t'r^ ology by which the capacity of Air Force units to perform OJT 
v '■ est imated . 



Mounting pressures to obtain the most cost-effective mix of resi- 
dent school training and OJT make it increasingly important for the 
Air Force to be able to determine the OJT capacities of its opera- 
tional units. At its present level of op-ration, OJT may be less 
costly or otherwise Pore efficient than evident school training for 
some AF?C (Air Force Specialty Code) skill categories in some types 
of Ai * force units. Yet, it would be unreasonable to replace all 
resident school training with OJT in such situations if doine so would 
cause the unit's OJT load to exceed enpaeif. At the extreme, oxces- 
-j i v a 0 JT might d e £ r a d e p r i m a r v mission per f o mane e b e i o w a c c e p t . i b 1 
standards. More typically, as training l-a;.ii<; : -a-;] : e , OJT costs are 
. I i l-j rise until the cost-e f fee tive no .-a; «. ■" f T relative to res id on 

school training has been e 1 : m ir.a ted . 

Clearly, Air Force decision-making concerning all ef these issues 
would be enhanced by a workable met hodo lory to estimate the capacitv 
of operational unit.: to conduct OJT. This report presents the results 
et a. i initial effort to develop and demonstrate such, a methodology. 

Thu.- , Sectirr. '.l. r ) provides background material concerning the 
aatare of OJT ana summarises pertinent elements of the OJT 11 te'rature, 
A conceptual model, for determining OJT capacity is then developed in 
Section '4.0. Next, Section 4.0 describes the selection of operational 
units as bases for empirically testing the model and discusses the 
data sources utilized for these empirical tests. The results of esti- 
mating the structural equations of the conceptual model of OJT capacit 



Stephenson, R. W. , and J. R. Burkett, On-the-Job Training in the 
Air Force: A Systems Analysis , AFHRL-TR-75-83 , AD-A036 206, Lowry AFB 
Colorado, Air Force Human Resources Laboratory, Technical Training 
Division, December 1975, p. 9. 
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for the selected operational units are reported in Section 5.0. 
Finally, Section 6.0 summarizes the conclusions derived in the study 
and presents specific recommendations for further development and 
implementation of the OJT capacity estimation methodology. 

The conceptual model of OJT capacity developed in this study is 
a concice description of the complex interrelationships among the 
level of mission performance, the quality of mission performance, the 
amount of training provided, and the quality of training achieved by 
an Air Force operational unit. Thus, the model incorporates represen- 
tations of the interdependence among these four attributes, the 
resources available to the unit, and pertinent external factors. The 
maximum capacity of a unit to conduct OJT is then determined as the 
maximum amount of training sustainable by the unit without compromising 
established standards for the level of mission performance, the quality 
of mission performance, and the quality of training. Similarly, the 
desirable OJT capacity of a unit is calculated as the maximum amount 
of training that the unit can conduct while maintaining a desirable 
balance among the level and quality of mission performance and train- 
ing. 

The model has been empirically tested by applying multiple linear 
regression analysis to estimate the basic structural equations of the 
model, to the extent possible, using existing data for six Air Force 
operational units. The operational units examined include four air- 
craft maintenance units responsible for servicing aircraft assigned 
to primary tactical flying missions , one transportation squadron 
responsible primarily for vehicle maintenance, and one supply squadron. 
In all cases, the data used in the analyses consist of measures 
routinely developed and maintained for the units by various Air Force 
offices. 

Despite the substantial limitations in the availability of suit- 
able data, the empirical analyses have revealed a persistent and strong 
inverse relationship between the amount of training conducted by a unit 
and the quality of mission performance attributable to the unit for all 
operational units studied. In addition, a perceptible, but weaker, 
correlation between the resources available to a unit. and the quality 
of the unit's mission performance has also been detected. Finally, 
the possibility that an association exists among the quality of train- 
ing provided in a unit, the training load of the unit, and conceivably, 
the resources available to the unit is also suggested by the empirical 
results. However, at least in the relatively low stress, peacetime 
situations examined in the study, no consistent interdependence is 
discernible among the level of mission performance achieved by a unit, 
the amount of training endue ted by the unit, and the availability of 
resources to the unit. 

The derivation of numerous statistically significant relationships, 
in spite of the relatively poor quality of the data obtainable from 
secondary sources and the absence of any indicators of some important 




explanatory variables in those sources, is very encouraging. This 
favorable result suggests that further development of the conceptual 
model of OJT. capacity formulated and preliminarily tested in this 
study can reasonably be expected to produce very useful methodologies 
for estimating OJT capacity. 

Specifically, the derivation of an implementable OJT capacity 
estimation methodology will require expanded exploratory research, 
supported by a methodical program of data acquisition and retention 
designed to provide a comprehensive set of indicators describing all 
of the variables contained in the conceptual model. Thus, data per- 
tinent to the construction of indicators of each variable in the 
conceptual model, possibly compiled at the level of individual work 
centers, should be collected and maintained for a representative 
sample of Air Force operational units. Empirical estimates of the 
coefficients of the model's basic structural equations should then 
be derived for each unit in the sample using these data. Next, the 
empirical results obtained for presumably similar units should be 
systematically compared and, to the extent possible, reconciled. 
Quantitative assessments of the OJT capacities of the units should 
then be produced on the basis of the reconciled coefficient estimates 
using the procedures formulated, but not applied, in this study. 
Finally, aggregation procedures should be developed, and empirically 
tested, to permit the calculation of OJT capacities for higher organi- 
zational levels than squadrons or flying wings. 

Detailed discussions of the conceptual model, the data bases, 
the analytic results, and the recommendations for further development 
are presented in the remainder of this report. 
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This ambiguity about the nature of OJT is reflected in the estab- 
lished procedures for the maintenance of personnel records. Upgrade 
training (UGT) is accorded formal OJT status. Personnel are explicitly 
enrolled in UGT; and training records are maintained throughout their 
enrollment. Conversely, qualification training seldom is formally 
recognized while it is being performed. Rather, such training typi- 
cally is recorded in a trainee's Job Proficiency Guide (JPG) only 
after it has been completed. 

Yet, in the context of an initial research effort to develop a 
methodology for estimating the capacity of operational units to per- 
form OJT, th so distinctions among the various types of training 
conducted by operational Air Force units are of minor importance. 
All such training imposes similar demands upon unit resources. 

• Qualified instructors and appropriate equipment must 
be provided for f, hands-on" proficiency training. 

» On occasion, equipment essential to primary mission 
performance must be released for use in training. 

• Work schedules must be arranged to bring trainees, 
experienced personnel, and suitable equipment together. 

• For certain specialized skills, funds must be budgeted 
for trainees 1 participation in field training detach- 
ment (FTD) programs. 

Thus, regardless of the precise form of training provided, the capacity 
of an operational unit to perform OJT is directly dependent upon the 
availability of training opportunities. Moreover, these training 
opportunities, in turn, will depend on the availability of unit 
resources in excess of those required for primary mission performance. 

Consequently, the provision of OJT and primary mission performance 
will typically be interdependent, largely through their competition for 
available resources. In addition, this interdependence generally will 
involve more than just an interaction between the level of mission 
performance achieved and the amount of OJT conducted. Rather, both 
the quality of mission performance attained and the quality of train- 
ing provided usually will also be affected. Thus, inherent in the 
nature of OJT is a complex interrelationship among the amount of 
training performed, the quality of training provided, the level of 
mission performance achieved, and the quality of mission performance 
attained; and the development of a methodology to estimate OJT capacity 
unavoidably involves analysis of this interrelationship. 
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2.2 Review of Pertinent Literature 



No previous study has directly examined the concept, of the capac- 
ity of operational units to perform OJT. Numerous studies of other 
aspects of OJT have been conducted i however. Many of these studies 
provide useful insights into the measurement of OJT capacity. 

The studies pertinent to the estimation of OJT capacity all 
address various elements of the cost-effectiveness of OJT programs. 
Thus, all of the pertinent studies can be related in some way to the 
basic concept that the cost-effectiveness of an OJT program can be 
evaluated by comparing the value added to the skills of the program's 
trainees with the costs of the resources used in the program. 

Most of these studies have, either explicitly or implicitly, used 
increases in trainee productivity as a measure of the value added by 
OJT, and have employed decreases in trainer productivity as a component 
of training costs. These valuation techniques have been applied most 
directly by Gay4 and by Gay and Nelson^ who have estimated the net 
value added by OJT as the present value of the differences over time 
between the actual productivity of trainees in the specialties in 
which they have been trained and the productivity of these trainees 
in their highest valued alternative military assignment. 

Productivity differences have been used somewhat less directly 
by Weiher and Horowitz^ and by 0 f Neill7 to measure OJT costs. Weiher 
and Horowitz include as an element of their cost estimates the value 
of foregone trainee output, which they measure as the difference over 
time between the trainees 1 wages and the value of their direct output. 
Similarly, O'Neill measures OJT costs in terms of the productivity 
foregone by replacing a journeyman with a trainee. 



Gay, R. M. , Estimating the Cost of On-the-Job Training in Mili- 
tary Occupations: A Methodology and Pilot Study , R-1351-ARPA, Santa 
Monica, California, The RAND Corporation, April 1974. 

^Gay, R. M. , and G. R. Nelson, Cost and Efficiency in Military 
Specialty Training , Santa Monica, California, The RAND Corporation, 
January 1974. 

6 Weiher s R. , and S. A. Horowitz, Formal and On-the-Job Training 
for Navy Enlisted Occupations , Professional Paper No. 83, Arlington, 
Virginia, Center for Naval Analyses, November 1971. 

7 0'Neill, D. M. , "Determinants of Labor Turnover Costs in the 
Military, " Study 4, Part 1, Volume I of Studies Prepared for the 
President's Commission on an All-Volunteer Araed Force , Washington, 
D.C. , U. S. Government Printing Office, November 1970. 
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Finally, Arzigian 8 strongly implies the use of productivity 
changes as a measure of the value added by OJT when he incorporates 
in his estimates of OJT costs the assumption that a trainee's effec- 
tiveness increases linearly over time from the conclusion of basic 
training through completion of OJT and attainment of the journeyman 
level. Thus, in combination, all of the studies discussed above 
clearly indicate both a direct relationship between the current pro- 
vision of OJT and future mission performance, and an inverse correla- 
tion between current OJT activities and current mission performance. 
These research results strongly support the conclusion reached in 
the preceding section that developing a methodology to estimate OJT 
capacity will necessarily involve examination of the complex inter- 
relationships between mission performance and the provision of OJT. 

Further evidence of the competitive relationship between current 
OJT efforts and current mission performance is provided by several 
studies that ^estimate the cost of OJT in terms of the value of the 
resources devoted to the provision of OJT. These studies, including 
research conducted by Dunham 9 and by Eisele, Bell and Laid law, 10 as 
well as the previously cited work by Arzigian 8 and by Weiher and 
Horowitz, 6 all identify the value of trainers' and supervisors' time 
devoted to OJT as key components of OJT cost. In the absence of 
corresponding increases in overtime, allocation of these resources 
to OJT will necessarily reduce the amount of experienced personnel 
effort available for primary mission performance. Thus, at the 
extreme, the excessive allocation of skilled personnel resources to 
conduct OJT can cause mission performance to decline below acceptable 
standards and thereby cause OJT capacity to be exceeded. 

Finally, a study by Lecznar 11 has examined the relative effective- 
ness of OJT and formal resident school training in terms of six criteria 
related to the primary mission performance of graduates of the two types 



Arzigian, S. , On-the-job Training Costs: An Analysis , U. S. Navy, 
Bureau of Naval Personnel, WRM 67-52, Washington, D.C., June 1967. 

9 Dunham, A. D. , Estimated Cost of On-the-job Training to the 3-Skill 
Level in the Communications Center Operations Specialty , AFHRL-TR-72-56 , 
AD-753 093, Lackland AFB, Texas, Air Force Human Resources Laboratory, 
Personnel Research Division, June 1972. 

10 Eisele, C. R. , T. R. Bell, and C. D. Laidlaw, Cost Analysis of Air 
Force On-the-Job Training; Development and Demonstration of a Method- 
ology , AFHRL-TR-78-88, Lowry AFB, Colorado, Air Force Human Resources 
Laboratory, Technical Training Division, May 1979. 

X1 Lecznar, W. B. , The Road to Work: Technical School Training or 
Directed Duty Assignment , AFHRL-TR-72-29 , AD-754 845, Lackland AFB, 
Texas, Air Force Human Resources Laboratory, Personnel Research Division, 
April 1972. 

u 
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of training. Specifically, the mission performance of technical school 
and OJT graduates are compared in terms of job difficulty, average task 
difficulty, number of tasks performed, job interest, utilization of 
talent and training, and overall job performance. This study, which 
reveals no significant differences between OJT and formal resident 
school training, suggests that measures of post-training performance 
represent potentially useful indicators of training quality. This 
insight has direct relevance to the formulation of quantitative indi- 
cators of training quality for use in empirically investigating the 
estimation of OJT capacity. 
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3.0 CONCEPTUAL MODEL OF OJT CAPACITY 



Based on the review of available literature and the- initial explor- 
ations into the nature of OJT, four key concepts have been identified 
as the basic elements of a conceptual model of OJT capacity. These 
concepts are 

• The level of mission performance. 

• The quality of mission performance. 

• The quality of training provided. 

• The amount of training provided. 

The predominant concern of any Air Force operational unit is the 
successful performance of its primary mission. Accordingly, any degra- 
dation in the level or quality of mission performance below acceptable 
standards cannot be tolerated. Training loads causing such degradation 
would be considered to exceed capacity. 

Similarly, the amount of training provided is meaningless if its 
quality is inadequate. Therefore, levels of training that cause train- 
ing quality to deteriorate below some minimum standard would also be 
considered to exceed capacity. 

These basic relationships are depicted graphically in Figure 1. 
CI, C2 , and C3 represent the maximum amounts of training sustainable 
without compromising the established standards for the level of mission 
performance, the quality of mission performance, and the quality of 
training, respectively. Thus, the OJT capacity indicated in this 
figure is C2. This amount of training is the maximum training load 
for which the level of mission performance, the quality of mission 
performance, and the quality of training all fulfill their minimum 
standards. Any greater amount of training would cause the quality 
of mission performance to decline to an unacceptable level.. 

This representation considerably oversimplifies the determination 
of OJT capacity, however. The level of mission performance, the quality 
of mission performance, and the quality of training arc unlikely to be 
independent. Rather, training quality and mission performance quality 
both can reasonably be expected to decline as the level of mission per- 
formance increases, at least at higher mission performance levels. 
Moreover, training quality and mission performance quality are also 
likely to be interrelated, although the nature of the correlation is 
more complex. Current mission performance quality shouid be positively 
influenced by the quality of previous training insofar as training 
improves product iv ity. Conversely , current training qua 1 i t y and 
current mission performance quality may be negatively related, since 
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FIGURE 1: Conceptual Model of OJT Capacity 
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their provision draws upon a common pool of resources. For example, 
skilled personnel performing training duty might improve training 
quality; while their absence from directly productive activities 
might reduce the quality of mission performance. 

In addition, other factors, such as resource availability, job 
characteristics, organizational considerations, and external influences 
can also affect the level and quality of both mission performance and 
training. A miDre detailed list of such factors is presented in Table 1. 

All of the propositions discussed should be incorporated in any 
comprehensive, operational model of OJT capacity. These propositions, 
and their interrelationships, can be conveniently summarized mathemat- 
ically in terms of the following system of equations: 



ML « f (TL, TQ p , R, E) (1) 

MQ = g (ML, TL, TQ, TQ , R, E) (2) 

P 

TQ = h (ML, TL, R, E) (3) 

where ML - the level of mission performance. 

MQ - the quality of mission performance. 

TL - the amount of training provided (the training 
load whose maximum attainable value defines 
the OJT capacity of the operational unit). 

TQ = Che quality of current training. 

TQp » the quality of previous training. 

R = the resources available to the operational unit. 

E « external, and other, f ors. 



The analysis presented previously in this section has examine.] in con- 
siderab?^e detail the anticipated nature of the interrelationships among 
the variables in these equations. These anticipated interrelationships 
can be restated conveniently in terms of the expected signs of the 
first-order partial derivatives of the equations. Specifically, the 
anticipated signs of those partial derivatives for which unequivocal 
expectations can be formed are 

» For equation dML/dTL < 0 , dML/dTQ >0, and dML/dR >0 . 

P 

e For equation (2), dMQ/dML < 0 , dMQ/dTL < 0 , dMQ/dTQ < 0 , 

dMQ/dTQ >0, and dMQ/dR> 0. 
P 

« For equation (3), dTQ/dML <0 , dTQ/dTL <0 , and dTQ/<3R>0. 



17 

o IS 

ERLC 



TABLE 1: Factors Influencing OJT Capacity 



• Resource availability 




Availability of needed quantity and quality of 
training facilities and equipment. 




Presence of professional training personnel. 




Presence of qualified instructors. 


- 


Availability of instructor time in excess of that 
required for non-training responsibilities, 
including the primary mission, record keeping, 
and administration. 




Influence of work shift scheduling and shop manning 
on the presence of skilled personnel needed for 
prof iciency training within a work center. 




Budgetary constraints on participation in field 
training detachment (FID) programs. 


• Job 


characteristics 




Difficulty of tasks to be trained. 




Frequency of occurrence of tasks requiring 
proficiency training. 


• Organizational considerations 




Command emphasis on the importance of training. 




Morale. 




Trainee characteristics, such as ability, 
knowledge, and motivation. 


• External influences 




Seasonality. 




Weather conditions, such as temperature and snow. 
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Finally, applying standards — determined by Air Force policy — speci- 
fying the minimum acceptable values of the level of mission performance 
(ML 0 ), the quality of mission performance (MQ°) , and the quality of 
training (TQ°) , the OJT capacity of the unit can be calculated as the 
maximum training load supportable by the unit. This calculation is 
depicted graphically in Figure I. Mathematically, it can be expressed 
as 

Maximize TL 

subject to ML>ML° 

MQ >MQ0 

and TQ >TQ° 

Similarly, if Air Force policy-makers can develop a functional 
relationship indicating the relative desirability of different com- 
binations of mission performance level, mission performance quality, 
training load, and training quality, the desirable OJT capacity of 
the unit can also be estimated. This estimation can be denoted math- 
ematically as 

Maximize D (ML, MQ, TL, TQ) 
TL 

subject to ML>ML° 
MQ > MQ° 
and TQ>TQ0, 

where D (ML, MQ, TL, TQ) is the unit's "desirability" 

function. 

Since, other things being equal, the unit prefers higher levels of each 
of the "desirability" function's independent variables to lower levels 
of the same variable, each of the function's first-order partial deriv- 
atives can reasonably be expected to be positive. 

The solution of this problem will indicate the optimal mission 
performance level, mission performance quality, training load, and 
training quality that can be achieved by the unit — relative to its 
own criterion of desirability — while maintaining acceptable values 
of mission performance level, mission performance quality, and training 
quality. It is conceivable that, in many situations, the minimum 
acceptability conditions will not impose binding constraints on the 
solution. Nevertheless, they must be included in the formulation of 
the problem to preclude any unacceptable degradations of the corre- 
sponding variables of primary interest. 
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4.0 FRAMEWORK FOR MODEL DEMONSTRATION 



To provide a basis for empirically testing the conceptual model 
of OJT capacity formulated in the preceding section, two preparatory 
tasks have been performed: 

• Selection of the specific Air Force operational units 
for which the application of the model has been demon- 
strated. 

• Identification of adequate sources of the various types 
of data required for application of the model for each 
selected unit. 

The outcomes of these two tasks are discussed in the following two 
subsections. 



4.1 Selection of Operational Units 



To assure a reasonably thorough assessment of the potential range 
of application of the OJT capacity estimation methodology, empirical 
tests of the conceptual model of OJT capacity have been conducted for 
two different broad classes of Air Force operational units: 

• Units performing maintenance on aircraft assigned to 
primary flying missions. 

• Units performing no aircraft maintenance activities. 

This grouping has been established in recognition of the clear distinc- 
tion between the direct association of aircraft maintenance activities 
with the level and quality of primary mission performance, and the much 
more indirect relationships of primary mission performance with most 
other types of activity. This distinction has clear practical signifi- 
cance for this study. Specifically, in aircraft maintenance units, OJT 
efforts can reasonably be expected to be directly relatable to measures 
of primary mission performance. Conversely, for other units, OJT activ- 
ities are likely, at best, to be relatable to instrumental indicators 
of unit performance that are believed to have some indirect ultimate 
influence on primary mission performance. Consequently, to investigate 
the potential range of application of the OJT capacity estimation meth- 
odology, empirical tests of the methodology must be conducted for both 
of these broad classes of Air Force operational units. 
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A total of six operational units have been examined in this study. 
Four are aircraft maintenance units, three attached to wings equipped 
with F-4 aircraft and one attached to a wing with RF-4 aircraft. 12 The 
fifth unit is a transportation squadron, responsible primarily for 
vehicle maintenance. The sixth is a supply squadron. 



4.2 Data Sources 



To perform a comprehensive empirical demonstration of the OJT 
capacity estimation methodology, six categories of data must be obtained. 
These categories consist of indicators of 

• The level of mission performance. 

• The quality of mission performance. 

• The amount of training provided. 

• The quality of training. 

• The availability of resources. 

• External and other factors. 

For the current study, no time or resources have been allocated for 
primary data collection efforts. Consequently, all data used in the 
empirical analyses have been obtained from existing secondary sources. 

Ideally, the data should describe activities and resources at the 
level of individual work centers. It is at this level that the inter- 
actions among resources, training loads, and mission performance that, 
in combination, determine OJT capacity can reasonably be expected to 
occur most strongly and directly. 

Unfortunately, none of the operational units examined in the study 
routinely develop and maintain the desired types of data at the work 
center level. Although it is relatively simple to obtain suitable 



Part of the reason for examining these four units has been to 
compare the relationships between training loads and mission perform- 
ance for two different types of aircraft maintenance organization: 
Production Oriented Maintenance Organization (POMO) and traditional 
organization. The results of this comparison are reported in P. J. 
DeLeo and T. R. Bell, Estimating the Capacity of Air Force Maintenance 
Organizations to Conduct On-the-Job Training , Lowry AFB , Colorado, 
Air Force Human Resources Laboratory, Technical Training Division, 
December 1979. 
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information about the current characteristics of any work center, it 
is virtually impossible to obtain this information for enough histor- 
ical time periods to permit empirical demonstration of the model. 

For example, in the aircraft maintenance units, the Maintenance 
Management Information Control System (MMICS) appears to be an ideal 
source of information. Among its many capabilities, MMICS contains 
data concerning "•••required training courses, inspection and special 
qualifications, OJT training information, training statuses, due and 
completed dates, and an individuals complete training history."^ 
Moreover, this information is recorded for virtually all ^raining 
activities pertinent to OJT, including "•••proficiency, qualification, 
certification, resident/FTD , dual channel OJT and locally established 
requirements. "13 However, MMICS does not systematically retain data 
describing a unit 1 s historical training activities. Whenever airmen 
are transferred to other units, all of their records are deleted from 
the data stored in MMICS for the units from which they are departing. 
Moreover, printed copies of MMICS data are retained as a safeguard 
against inadvertent information losses for at most 3 months. Conse- 
quently, MMICS cannot provide a sufficiently long sequence of histo- 
rical training and mission performance information to permit its use 
as a data source in this study. 

As a direct result of such data limitations, it has been impos- 
sible to perform any empirical demonstrations of the OJT capacity 
estimation methodology at the work center level. Rather, the method- 
ology has been applied at the squadron level for the transportation 
and supply units, and at the wing level for the aircraft maintenance 
units. Only at these higher organizational levels could a sufficient 
amount of existing historical data be obtained to permit empirical 
analysis. 

For the aircraft maintenance units, the data used to demonstrate 
the OJT capacity estimation methodology have been assembled from six 
different sources. Most of these information sources are printed 
summaries published by various wing-level analysis and evaluation 
offices. 

Monthly Maintenance Summary, Part I, prepared by the 
wing maintenance analysis office, provides numerous 
monthly indicators of wing mission performance. 
Historical records of the indicators are reported in 
each summary for as many as 12 consecutive months. 
Moreover, because this information is submitted to the 
analysis staff at MAJCOM (Major Command) headquarters, 
all wings employ the same format in this document. 



department of the Air Force, Maintenance Management Information 
Control System (MMICS) , Volume I, AFM 66-278 (C12), Washington, D.C. , 
November 1977, p. 10-1. 
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• Monthly Maintenance Summary, Part IV, 1A also compiled 
by the maintenance analysis office, contains a summary 
of quality control evaluations. Two types of evalua- 
tions are reported: personnel evaluations appraising 
the adequacy of task performance as determined by 
directly observing qualified airmen performing the 
task, and task evaluations assessing the completeness 
and correctness of task performance after its execu- 
tion. To the extent that maintenance task performance 
contributes to primary mission performance, these 
evaluations can be used as indicators of the quality 
of mission performance. Alternatively, to the extent 
that successful on-the-job task performance indicates 
adequacy of training, the evaluations can be inter- 
preted as measures of training quality. In either 
event, this information represents potentially 
pertinent data for the study. 

• Records maintained by the training management office 
specify the amount of formally monitored training 
conducted in the aircraft maintenance units. In par- 
ticular, these records report UGT enrollments and 
completions. However, the training management offices 
of different bases employ different procedures concern- 
ing the retention of historical information. 

• The Daedalian Monthly Maintenance Award Report, which 
also publishes monthly UGT enrollment, can, fortunately, 
normally be used to supplement the other training 
records to obtain an uninterrupted historical sequence 
for UGT enrollment. This report also contains infor- 
mation about a variety of other aircraft maintenance 
activities and, hence, can often provide values found 

to be missing in other data sources. 

• The Organizational Manning Report, prepared by the 
programs and mobility office, provides an exact record 
of actual personnel assignments in individual work 
centers. This report identifies the work center, 
duty assignment, and Primary Air Force Specialty Code 
(PAFSC) for each airman assigned to the unit; and it 
indicates the authorized manning levels for each work 
center. 



Monthly Maintenance Summary, Parts II and III* contain more 
detailed information about the maintenance of individual aircraft and 
specific component systems. Thus, these reports directly reflect the 
performance of individual work centers. However, as explained previ- 
ously, the absence of comparable data about training activities pre- 
cludes the use of this information. 

24 

O 

ERLC 



• Records maintained by the Wing Historian can occasion- 
ally be used to provide information missing from other 
data sources. However, most of the information com- 
piled by this office pertains to the entire wing, but 
does not describe individual squadrons or work centers. 

Obviously, many of these data sources just described contain infor- 
mation that overlaps data in other sources. A complete compilation of 
the potentially pertinent indicators reported in each cited data source 
is presented in Table 2. Nevertheless, undesirable missing information 
remains for many important variables and reduces the usable size of the 
final data base. For some indicators, 50 consecutive monthly observa- 
tions are available; while for other variables, only four observations 
have been obtained. 

For the transportation and su; uadrons, fewer sources of 

pertinent information have been fo'. nevertheless, the identified 

sources have provided uninterrupte .storical data series for all 
variables of interest spanning 2 msecutive months. 

© For the transportation squadron, the VIMS (Vehicle 
Integrated Management System) Monthly Analysis, 
prepared by the maintenance analysis branch of the 
squadron, furnishes several monthly measures of the 
performance of the squadron's vehicle maintenance 
mission. Additional unpublished vehicle maintenance 
performance indicators are also prepared by and 
available from the squadron's maintenance analysis 
branch. 

a For the supply squadron, internal reports compiled 

by its analysis office provide many monthly indicators 
of the level and quality of the squadron's logistic 
efforts. For each of the squadrons, historical values 
of the indicators are presented in each report for as 
many as 12 consecutive months. 

• Records of the amount of formally monitored training 
conducted in the two squadrons are maintained by the 
OJT office of the Consolidated Base Personnel Office. 
These records indicate, for each squadron, the total 
monthly training loads and upgrades for each skill 
level, and the overall amounts of UGT and retraining 
conducted in each month. 

© Additional records compiled by the OJT office specify 
the actual number of personnel assigned to each 
squadron in each month. The number of personnel 
authorized for each squadron throughout each quarter 
is also available through the Consolidated Base 
Personnel Office. 
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TABLE 2: Aircraft Maintenance Unit Data Sources 



Data Source 
(Docunent 
and/or Office) 



Data Item 
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rrlmary Air Force specialty coda. 
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• Data concerning monthly weather conditions have been 
obtained from the base weather station. These data 
describe the typical base weather situation in terms 
of monthly maximum, minimum, and average temperature, 
snowfall, and precipitation levels. 

Complete compilations of the potentially pertinent indicators 
available from each cited data source are presented for the transpo 
tation and supply squadrons in Tables 3 and 4, respectively. 
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TABLE 3: Transportation Squadron Data Sources 



Data Sources 
>w (Document 

and/or Office) 

Date Ifcflft 


VIMS Monthly Analysis 


Maintenance Analysis 
Branch, Transpor- 
tation Squadron 


OJT Office, 
Consolidated Base 
Personnel Office 


Consolidated Base 
Personnel Office 


Base Weather Station 


tun uvui 9 


X 










Estimated direct labor hours required 
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Actual direct labor hours applied 


z 










UGT enrollment 
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OJT upgrades to 3, 5, and 7 levels 
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Personnel assigned 
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X 




Personnel authorized 










Assigned personnel fully trained 












for grade 






z 






Average monthly temperature 










Z 


Average monthly snowfall 










Z 


Average monthly precipitation 










z 



Vehicle down for maintenance* 



'8 
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TABLE A: Supply Squadron Data Sources 



^v*. uaca sources 
^^^^s^ (Document 

^* s *w And /or Office} 

Data Item 


Analysis Office, 
Supply Squadron 


OJT Office, 
Consolidated Base 
Personnel Office 


Consolidated Base 
Personnel Office 


Base Weather Station 


Reverse post rate 


X 








Inventory adjustments 


X 








RCA* units over and short 


X 








Backorders 


X 








EOQ b units over and short 


X 








Total audltable docunents 


X 








Line Items Issued 


X 








Line Items Issued and backordered 


X 








RCA a units 


X 








Total transactions 


X 








UGT enrollment 




X 






OJT upgrades to 3, 5, and 7 levels 




X 






Personnel assigned 




X 


X 




Personnel authorized 








Assigned personnel fully trained for grade 




X 




X 


Average monthly temperature 








Average monthly snowfall 








X 


Average monthly precipitation 








X 



a Repair cycle assets. 
^Economic order quantity. 
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5.0 EMPIRICAL RESULTS 



Despite the limited availability of suitable data from existing 
sources, a persistent and strong relationship between mission perform- 
ance and training load has been estimated for all operational units 
studied. The derivation of statistically significant relationships 
in spite of the relatively poor quality of the obtainable indicators 
and the absence of some important explanatory variables is most encour- 
aging. This favorable result suggests that further study can reason- 
ably be expected to produce very useful methodologies for estimating 
OJT capacity. 

The data limitations encountered in the study have caused four 
basic problems in empirically demonstrating the application of the 
methodology: 

• The available indicators of mission performance seldom 
are unambiguous measures of either the level or the 
quality of performance. Consequently, the empirical 
results derived for these indicators are often amenable 
to varying and inconclusive interpretations. 

• Satisfactory indicators of the quality of training are 
generally unavailable. Hence, no relationships respon- 
sive to the quality of training have been derived. 

• The existing indicators of training load essentially 
measure only gross UGT activity. No measures of other 
types of training, or of trainees' progress within UGT, 
are available. 

• Indicators of other resources and external effects are 
restricted to measures of enlisted personnel strength 
and weather conditions. 

If more directly applicable data were made available — by retaining 
data now routinely discarded or by systematically collecting new data 
— these difficulties would be sharply reduced. 

Nevertheless, the empirical results collectively demonstrate a 
convincing relationship between the mission performance and training 
loads of Air Force operational units. These results, developed by 
applying multiple linear regression analysis to the conceptual model 
of OJT capacity formulated in Section 3.0, are presented in the 
remainder of this section. 

The results for the three different types cf operational units 
appear in the order of increasing complexity. This order of presenta- 
tion will permit the reader to comprehend the less complex results 
fully before being confronted with the results that are more difficult 
to interpret. 
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Thus, the results derived for the transportation squadron appear 
in Section 5.1; the results for the supply squadron are reported in the 
following section; and the results for the aircraft maintenance units 
are sumnarized in Section 5.3. The transportation squadron offers only 
one good measure of mission performance, and the available data for the 
explanatory variables are limited also. The supply squadron provides 
two indicators of mission performance. However, these measures are not 
very well defined and, hence, present problems in interpretation of the 
empirical results. Finally, the aircraft maintenance units furnish the 
largest amounts of data, but also entail the greatest difficulties in 
interpretation. 



5.1 Transportation Squadron 



Due to data limitations, the only equation from the conceptual 
model of OJT capacity that has been estimated for the transportation 
squadron is the equation describing the quality of mission performance 

MQ - g (ML, TL, TQ, TQ p , R, E) 

where, as before, 



MQ 




the quality of mission performance. 


ML 


m 


the level of mission performance. 


TL 


m 


the amount of training provided. 


TQ 


m 


the quality of current training. 


TQ 

P 


m 


the quality of previous training. 


R 


m 


the resources available to the squadron. 


E 


m 


external, and other, factors. 



In addition, because no suitable data have been found measuring either 
the level of mission performance or the quality of training achieved in 
the squadron, the relationship actually examined has been 

MQ - G (TL, R, E) 

Four different formulations of this basic relationship in which mission 
performance quality is associated with training load, resource avail- 
ability, and external factors have been estimated for the transportation 
squadron. 
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Only one acceptable indicator of the dependent variable, the 
quality of mission performance , has been uncovered for this squadron. 
This indicator is the total number of hours that vehicles have been 
down for maintenance by the transportation squadron during each month 
(VDMHRS). It represents a conceptually .satisfactory measure of the 
quality of mission performance since, other things being equal, the 
greater are the number of hours that vehicles have been done for 
maintenance by the squadron, the less productive the squadron's main- 
tenance efforts must have been. Unfortunately, other things are rarely 
equal. The number of miles that vehicles are driven, the rate at which 
vehicles arrive for maintenance, and the difficulty of the maintenance 
required by the newly arrived vehicles all vary from month to month. 
Consequently, VDMHRS unavoidably reflects these, and other, aspects 
of changes in the workload confronting the squadron. Moreover, since 
no data have been found describing any such external factors, it is 
impossible to adjust for these effects empirically. Rather, the 
possible influence of any excluded explanatory variables should be 
taken into consideration when interpreting the empirical results. 

The explanatory variables entered into the estimated relationships 
include indicators of the amount of training provided (TL), squadron 
resouL.es (R) , and some external factors (E). 

The basic variable used as an indicator of the amount of training 
provided in the squadron is number of enlisted personnel enrolled in 
UGT in each month (TLOAD) . This variable captures most of the squadron's 
training load. However, it does not reflect informal OJT activities 
such as qualification training, nor does it account for any progress 
achieved by trainees as their OJT proceeds. In two of the relationships 
estimated for this squadron, TLOAD is entered directly as an explanatory 
variable. In the remaining two formulations, TLOAD has been divided by 
the mn?b?r of enlisted personnel assigned to the squadron to produce a 
new explanatory variable, the proportion of the squadron's enlisted 
person n ' enrolled in UGT in each month (RELTLD). 

Only one basic variable has been found to measure the resources 
available to the squadron. This variable specifies the number of 
enlistea personnel assigned to the squadron in each month. As indi- 
cated previously, it has been used as the denominator of the explana- 
tory variable RELTLD in two of the estimated relationships. Moreover, 
in one other formulation, it has been divided by the number of enlisted 
personnel authorized for the squadron in each month to create an addi- 
tional explanatory variable, the proportion of the squadron's author- 
ized enlisted strength actually assigned to the squadron in each month 
(RELSTR) . 



Finally, two somewhat crude measures of monthly weather conditions 
have been included as indicators of external factors. These variables 
are the average snowfall in each month (SNOW) and the deviation of the 
average temperature in each month from the average annual temperature 
at the Air Force base housing the squadron (DEVTMP) . 

Thus, assuming a basic linear relationship, the general form of 
the equation actually estimated for the transportation squadron is 



MQ 


= a + b TL + c R + d E 




» VDMHRS. 


TL 


= TLOAD or RELTLD. 


R 


- RELSTR. 


E 


« SNOW and /or DEVTMP. 



Table 5 presents the empirical results obtained by applying mul- 
tiple linear regression analysis to derive estimates of four different 
formulations of this general equation. These represent all versions 
of the general equation for which all estimated coefficients are statis- 
tically significant at the 5 percent level. All attempts to establish 
versions of the equation containing the constant term, a, have produced 
statistically insignificant estimates of the constant. Consequently, 
none of the formulations reported in Table 5 contains a constant term. 

Despite all of the inadequacies in the data, these results uni- 
formly reveal a statistically significant positive relationship between 
the level of training provided in the transportation squadron and 
VDMHRS. Although the overall quality of the estimated relationships, 
as measured by R 2 , is not exceptional, the generally high values of 
the t-statistics!5 associated with the training load variables clearly 
and strongly indicate that increases in the amount of training con- 
ducted in the transportation squadron are significantly correlated with 
reductions in the quality of the squadron 1 s mission performance. In 
addition, the results suggest that RELTLD, the unit's training load 
relative to its assigned enlisted strength, provides a better descrip- 
tion of this relationship than does TLOAD, the absolute amount of 
training conducted in the squadron. 



Each t-statistic reported in Table 5 indicates the number of 
standard deviations by which the corresponding estimated regression 
coefficient differs from zero. The significance level associated with 
the t-statistic, therefore, specifies the probability that the esti- 
mated coefficient would have been derived empirically if the true 
value of the coefficient were zero. 
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TABLE 5: Bnpirical Results of Transportation Squadron Regressions 



Regression Coefficients (and t-statistics) 


R 2 


R2 


F-statistic 
(and 

degrees of 
freedom) 


TL 


R 


E 


TLOAD 


RELTLD 


RELSTR 


SNOW 


DEVTMP 


301.3 
(1.80)a 




6,219.3 
(1.73) a 


1 ,230.2 
(2.09) a 


456.3 
(3.02) a 


0.331 


0.164 


1.86 
(4,20) 


492.7 
(13.12) 3 






1,620.9 
(3.29) a 


565.0 
(4.68)a 


0.437 


0.338 


4.14 
(3,20) a 




45,821.4 
(12.63)a 




1,564.7 
(3. 04) a 


521.5 
(4.08) a 


0.316 


0.196 


2.47 
(3,20)b 




55,294.9 
(24.68)a 






184.2 
(2.41) a 


0.257 


0. 174 


2.94 
(2,20)b 



^Significantly different from 0 at the 5 percent level. 
^Significantly different from 0 at the 10 percent level. 
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5.2 Supply Squadron 



As In the case of the transportation squadron , the lack of suit- 
able data for the supply squadron permits the estimation of only one 
equation from the conceptual model of OJT capacity formulated in 
Section 3.0. Moreover , as before, the only relationship that can be 
empirically investigated is the equation describing the quality of 
mission performance 

HQ - g (ML, TL, TQ, TQ p , R. E). 

However, for the supply squadron, no data have been located describing 
either the level of training quality achieved or any external factors 
likely to affect supply operations. Consequently, the relationship 
actually analyzed for this squadron has been 

MQ - G (ML, TL, R). 

Fifteen different formulations of this basic relationship, in which 
mission performance quality is associated with mission performance 
level, training load, and resource availability, have been estimated 
for the supply squadron. 

Since no, single definitive measure of the dependent variable, the 
quality of mission performance, has been uncovered, two different mea- 
sures considered to be reasonable indicators of poor mission perform- 
ance quality have been employed as dependent variables. The first 
variable measures adjustments in recorded inventory balances (INVADJ); 
while the second measures reverse postings (RPOST) — computer entries 
that reverse previous erroneous entries. Both variables measure, 
basically, personnel errors. Hence, both should capture some aspects 
of the quality of mission performance. Yet, they reflect neither the 
activities of all personnel in the squadron, nor all facets of the 
overall performance of the unit. 

Three fundamental types of explanatory variables enter into the 
estimated relationships, including indicators of the level of mission 
performance (ML) , the amount of training provided (TL) , and squadron 
resources (R). 

Three different variables have been tested as indicators of the 
level of mis 8 ion performance: 

• ISSUED: the number of items issued by the squadron 
in each month. 

• TRANS: the number of computer transactions processed 
by the squadron in each month. 

• DCC: the number of auditable documents generated by 
the squadron in each month. 




None of these variables represents an ideal measure of the level of 
mission performance * but each does capture some aspect of the amount 
of supply services provided. 

Once again, the basic variable used to indicate the amount of 
training conducted in the squadron is the number of enlisted personnel 
enrolled in UGT in each month (TLOAD) . In seven of the relationships 
estimated for this unit, TLOAD is entered directly as an explanatory 
variable. In seven others, TLOAD has been divided by the nimber of 
enlisted personnel assigned to the squadron to create a new explana- 
tory variable, the proportion of the enlisted personnel enrolled in 
UGT each month (RELTLD) • Finally, to exanine the effect on mission 
performance quality of changes in the amount of training conducted 
over time, one relationship has been estimated using the difference 
between the squadron's training loads in consecutive months (DELTLD) 
as an explanatory variable. 

Only one variable has been discovered to measure the resources 
available to the squadron. This variable specifies the number of 
enlisted personnel assigned to the unit and, as indicated previously, 
has been used as the denominator of the variable RELTLD in seven of 
the estimated relationships. 

Therefore, assuming again a linear relationship, the general form 
of the equation actually estimated for the supply squadron is 

MQ - a+bML + cTL 

where MQ - INVADJ or RPOST. 

ML - ISSUED, TRANS , or DCC. 

TL - TLOAD, RELTLD > or DELTLD. 

Tables 6 and 7 present the empirical results obtained by applying 
multiple linear regression analysis to generate estimates of 15 differ- 
ent formulations of this general equation. These formulations include 
all versions of the general equation for which all estimated coeffi- 
cients are statistically significant at the 5 percent level. In 
addition, to facilitate comparison of the results obtained for the 
two alternative indicators of the quality of mission performance, the 
tables present the estimates derived for each of these dependent vari- 
ables using as explanatory variables all pairvise combinations of 
TLOAD or RELTLD with ISSUED, TRANS, or DCC. The results derived using 
INVADJ as the dependent variable appear in Table 6; while Table 7 dis- 
plays the results obtained using RPOST as the dependent variable. 




TABLE 6: Bnpirical Results of Supply Squadron 

Regressions with INVADJ as Dependent Variable 



Regression Coefficients (and t-statistics) 


R2 


R 2 


F-statistic 
(and degree 
of freedom) 


Constant 


TL 


ML 


TLOAD 


RELTLD 


ISSUED 


TRANS 


DCC 


-255.7 


10.96 
(2.l4) a 










0.203 


0.114 


4.57 
(l,18) a 


-85. A 


11.34 
(2.16) a 




-0.187 
(-0.61) 






0.220 


0.082 


2.39 
(2,17) 


-170.7 


11.24 
(2.07) a 






-0.0007 
(-0.22) 




0.205 


0.065 


2.19 
(2,17) 


-347.7 


10.91 
(2.07) a 








0.0027 
(0.19) 


0.204 


0.064 


2.18 
(2,17) 


-303.8 




4188.9 
(2.20) a 








0.212 


0. 124 


4.84 
(l,18) a 


-130.5 




4344.7 
(2.23) a 


-0. 193 
(-0.63) 






0.230 


0.094 


2.54 
(2,17) 


-205.1 




4323.7 
(2.14) a 




-0.0009 
(-0.26). 




0.215 


0.077 


2.33 
(2,17) 


-387.9 




4170.5 
(2.13) a 






0.0025 
(0.17) 


0.213 


0.075 


2.30 
(2,17) 



a Significantly different from 0 at the 5 percent level. 
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TABLE 7: Empirical Results of Supply Squadron 

Regressions with RPOST as Dependent Variable 



Regression Coefficient (and t-statistics) 


R 2 


R2 


F-statist 
(and degi 
of freed( 


Constant 


TL 


R 


TLOAD 


RELTNG 


DELING 


ISSUrD 


TRANS 


DCC 


388.1 


-2.46 
(-1.38)° 






0.219 

/ 1 1 1 \ St 

(2. ll) a 






0.252 


0.120 


2.86 
(2,17] 


148.1 


-3.50 
(-2.52) a 








0.0038 
(4.46) a 




0.566 


0.489 


11.06 

(2,17, 


45.8 


-2.29 
(-1.63)b 










0.0159 
(4.13) a 


0.529 


0.449 


9.54 
(2,17] 


386.4 




-868.4 
(-1.30) 




0.219 
(2.09) a 






0.243 


0.109 


2.72 
(2,17] 


151.2 




-1281.5 
(-2.44) a 






0.0039 
(4.43) a 




0.558 


0.480 


10.73 
(2,17] 


43.7 




-809.4 
(-1.53)b 








0.0159 
(4.10) a 


0.521 


0.437 


9.25 
(2,17] 


174.1 






-12.31 
(-3.56) a 


0.265 
(2.83) a 






0.522 


0.432 


8.73 
(2,16] 



iignificantly different from 0 at the 5 percent level. 
Significantly different from 0 at the 10 percent level. 
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The regression coefficients and t-statistics reported in Table 6 
indicate that IHVADJ is uniformly positively related to the amount of 
training provided in the unit, although the overall quality of the 
estimated relationships, as measured by R , is not exceptional. These 
empirical results also suggest that the incidence of personnel errors 
leading to inventory adjustments is not influenced significantly by 
the volume of work processed by the squadron. 

Conversely, as revealed in Table 7, RPOST is consistently nega- 
tively related to all indicators of the level of training conducted 
by the squadron. Although this outcome is contrary to prior expecta- 
tions, two possible explanations for this empirical result appear 
reasonable: either trainees perform relatively menial tasks, such 
as entering Information into a computer, more carefully than do expe- 
rienced personnel, or else trainees may be more closely supervised 
than are the fully trained airmen. The results presented in Table 7 
also indicate that the Incidence of personnel errors resulting in 
reverse postings is significantly positively related to the amount 
of work processed by the; squadron. In fact, the t-statistics suggest 
that, in general, RPOST is more strongly influenced by workload cotv- 
slderatlons than by training load effects. 

To provide some perspective tc these apparently contradictory 
sets of empirical results, observe that, in contrast to RPOST, IHVADJ 
reflects a much t >ader range of personnel errors, including errors 
associated with the more complex, less menial supply operations. 
Thus, IHVADJ is a more comprehensive indicator of the quality of 
mission performance than is RPOST; and the results reported in Table 6 
represent a more thorough and convincing test of the overall relation- 
ship between mission performance quality and squadron training loads 
than do the results derived using RPOST as the dependent variable. 



5.3 Aircraft Maintenance Units 



The four aircraft maintenance units have presented the greatest 
opportunity in this study for empirically testing all components of 
the conceptual model of OJT capacity. Despite considerable variabil- 
ity in data availability and quality, reasonable indicators are avail- 
able for the level of mission performance, the quality of mission 
performance, the amount of training conducted, and the quality of 
training provided in each of the units. 

Consequently, all three basic equations in the conceptual model 
of OJT capacity have been examined for the aircraft maintenance units 

ML - f (TL, TQ p , R, E) 

MQ - g (ML, TL, TQ, TQ p , R, E) 

TQ - h (ML, TL, R, E) 
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where, once again, 



ML 




the 


Iava i of fflitflion oerf o nuance. 


MO 




the 


fliral'lt'V of mission Derformance. 


m 


OH 


the 


quality of current training. 






the 


quality of previous training. 


Tl 




the 


amount of training provided. 


R 




the 


resources available to the unit. 


E 




external, and other, factors. 



Only one usable indicator of the level of mission performance has 
been u ncov e re d for the aircraft maintenance units. This indicator is 
the ratio of the total number of f lying-hours generated by the aircraft 
maintained by a unit in each month, to the total number of person-days 
for which enlisted personnel have been assigned to the unit during the 
month (FLYHRS). 



In contrast, 17 different indicators of the quality of mission 
performance have been developed. To simplify interpretation of the 
empirical results, each indicator has been formulated so that its 
value Increases as mission performance quality Improves. The indi- 
cator heir interpretations, and their methods of calculation are 



HGBORTS: the absence of flights aborted before takeoff 
■ 1.0 - (the number of ground aborts in each 
month/ the number of sorties flown during the 
month) • 

NABORTS: the absence of flights aborted after takeoff 
- 1.0 - (the number of air aborts in each 
month/the number of sorties flown during the 
month) • 



NBORTS: the absence of aborted flights 

- 1.0 - (the total number of air and ground 
aborts in each month/the number of sorties 
flown during the month). 

NMNTC: the absence of maintenance-related sortie 
cancellations 

■ 1.0 - (the number of sortie cancellations 
related to maintenance in each month/ the 
umber of sorties flown during the month). 



• NMATTR: 



• NFAIL: 



c RR3HR: 



• RR6HR: 



• MEFFC: 



• ONTIME: 



SORTYS: 



* NNMCU: 



the absence of maintenance attrition 

- 1.0 - maintenance attrition rate in each 
month. 

the absence of aircraft and component failures 

- 1.0 - (the number of aircraft and component 
failures in each month/ the number of sorties 
flown during the month). 

the 3-hour recovery rate 

« the number of aircraft repaired and ready 
for flight within 3 hours during each month/ 
the number of aircraft requiring maintenance 
in the month. 

the 6-hour recovery rate 

- the number of aircraft repaired and ready 
for flight within 6 hours during each month/ 
the number of aircraft requiring maintenance 
in the month. 

the maintenance scheduling effectiveness rate 
» a maintenance task accomplishment index 
developed by the Air Force. 

the ability to take off on schedule 

- 1.0 - (the number of late takeoff s in each 
month/ the number of sorties flown during the 
month) • 

the ability to fulfill the programmed flying 
schedule 

» the number of sorties flown as scheduled 
during each month/ the number of sorties 
scheduled for the month. 

the proportion of aircraft time during which 
available aircraft are free of essential 
unscheduled maintenance requirements 
■ 1.0 - (the total number of hours for which 
aircraft are not mission capable due to unsched- 
uled maintenance in each month/ the total number 
of hours for which aircraft are available to the 
unit during the month). 
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• NMNCUS: the proportion of aircraft time during which 

available aircraft are free of essential scheduled 
or unscheduled maintenance requirements 
■ 1.0 - (the total number of hours for which air- 
craft are not mission capable due to scheduled or 
unscheduled maintenance in each month/ the total 
number of hours for which aircraft are available 
to the unit during the month) • 



• NPMCM: the proportion of aircraft time during which 

available aircraft are f*:ee of needed, but not 
essential, maintenance requirements 
■ 1.0 - (the total number of hours for which air- 
craft are partially mission capable, but requiring 
maintenance, in each month/ the total number of 
hours for which aircraft are available to the 
unit during the month). 



• NPCNM; the proportion of aircraft time during which 

available aircraft are completely free of 
maintenance requirements 

■ 1.0 - (the total number of hours for which 
aircraft are either not mission capable due to 
maintenance, or partially mission capable but 
requiring maintenance, in each month/ the total 
number of hours for which aircraft are available 
to the unit during the month). 

• NRCWU: the absence of recurring equipment failures 

■ 1.0 - (the number of maintenance writeups that 
recur within the subsequent three sorties, but 
do not occur on consecutive sorties, in each 
month/ the total number of maintenance writeups 
in the month) . 



• BREPR: the base repair rate 

■ an index developed by the Air Force describing 
the proportion of maintenance tasks performed on 
location without external assistance in each 
month. 



Although each of these indicators reflects some aspects of the 
quality of mission performance attributable to the aircraft maintenance 
units, none represents a comprehensive and conclusive measure of mission 
performance quality. Consequently, all 17 indicators have been used in 
the empirical testing of the conceptual model of OJT capacity for the 
aircraft maintenance units. 
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Similarly, three indicators of the quality of training provided by 
the units have been included in the analysis. One of these indicators 
assesses the quality of the maintenance work performed by trained per- 
sonnel; and two appraise the quality of the maintenance practices 
employed by such personnel while performing the work- The indicators, 
their interpretations, and their methods of calculation are 

e QINSPC: the satisfactory quality of completed maintenance 
tasks 

- the number of quality verification inspections 
in which completed tasks are judged satisfactory 
in each month/ the number of quality verification 
inspections conducted during the month. 

• PINSPC: the satisfactory quality of maintenance task 

performance 

« the number of personnel inspections in which 
personnel are observed performing tasks satis- 
factorily in each month/the number of personnel 
inspections conducted during the month, 

• SAFEE: the safe performance of maintenance tasks 

- 1.0 - (the number of safety violations 
detected during personnel inspections in each 
month/the number of personnel inspections conr- 
ducted during the month). 

All of these indicators are measures of on-the-job performance subse- 
quent to training. Thus, while they are proffered as assessments of 
training quality, they can be interpreted equally justifiably as 
indices of mission performance quality. This ambiguity is unavoidable, 
and should be taken into consideration when interpreting the empirical 
results. 

The amount of training provided in the aircraft maintenance units 
is also represented by three indicators: 

• PTRN: the amount of proficiency training conducted 

- the number of trainees enrolled in UGT to 
the 5-skill level in each month/the number of 
5-, 7-, and 9-skill level personnel assigned 
to the unit during the month. 

• STRN: the amount of supervision training conducted 

« the number of trainees enrolled in UGT to 
the 7-skill level in each month/the number of 
7- and 9-skill level personnel assigned to the 
unit during the month. 
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• RE57: the amount of formal training conducted 

■ the number of trainees enrolled in UGT to 
the 5-skill level and to the 7- skill level 
in each month/ the total number of enlisted 
personnel assigned to the unit during the 
month. 



As before, these indicators constitute incomplete measures of the CUT 
loads borne by the units because they fail to reflect informal OJT 
activities such as qualification training, and they fail to account 
for any progress achieved by trainees during OJT. 

Next, four indicators reflecting the availability of resources to 
the units have been derived: 



• LAMN: shortages in the assignment of enlisted per- 

sonnel 

- 1.0 - (the number of enlisted personnel 
assigned to the unit in each month/the number 
of enlisted personnel authorized for the unit 
in the month). 

• LSKILL: shortages in the availability of skilled per- 

sonnel 

- 1.0 - (the average skill level of the enlisted 
personnel assigned to the unit in each month/the 
average skill level of the enlisted personnel 
authorized for the unit in the month). 



• INEXP: the relative inexperience of the enlisted 

personnel 

m the number of 3- ski 11 level personnel assigned 
to the unit during each month/the number of 5-, 
7*, and 9-skill level personnel assigned to the 
unit during the month. 

• TRNRS: the availability of professional training 

personnel 

- the number of personnel with PAFSC 75XXX 
assigned to the unit in each month/the total 
number of enlisted personnel assigned to the 
unit in the month. 



To facilitate direct comparisons with the training load indicators, 
the first three resource availability indicators have been designed to 
describe deficiencies in the availability of resources to the aircraft 
maintenance units. Only TRNRS represents a direct measure of the avail- 
ability of resources to the units. 
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Finally, to reflect key aspects of the effects of weather condi- 
tions on mission performance and training quality, a dummy variable 
distinguishing the cold weather months from the other, more temperate 
months has been included as an indicator of external factors. This 
variable, SEASON, is set equal to one for each month between November 
and March, and to zero for each other month, to compensate for the 
principal seasonal variations in the level and quality of mission 
performance and the quality of training. 

Due primarily to the large number of indicators of the quality 
of mission performance identified for the aircraft maintenance units, 
no attempt has been made, during the empirical testing of the conceptual 
model of OJT capacity for these units, to investigate the direct inter- 
actions among the level of mission performance, the quality of mission 
performance, and the quality of training. Consequently, the relation- 
ships actually examined for the aircraft maintenance units have been 

ML * F (TL, R, E) 

MQ - G (TL, R, E) 

TQ = H (TL, R, E) 

Moreover, assuming basic linear relationships, the general forms of the 
equations actually estimated for these units are 

ML = a + bTL + cR + dE 

MQ - a 1 + b 1 TL + c' R + d 1 E 

TQ = a" + b" TL + c" R + d" E 

where ML = FLYHRS . 

MQ » NGBORTS, NABORTS, NBORTS, NMNTC, NMATTR, NFAIL, 
RR3HR, RR6HR, ONTIME, SORTYS, MEFFC, TOMCU, 
NNMCUS, NPMCM, NPCNM, NRCWU, or BREPR. 

TQ * QINSPC, PINSPC, or SAFET. 

TL » PTRN, STRN, and /or RE57. 

R » LAMN, LSKILL, INEXP, and/or TRNRS. 

E « SEASON. 

However, inconsistencies among the time periods for which different data 
series are available, combined with extensive multicollinearity among 
the explanatory variables, have precluded examination of all conceivable 



formulations of the three basic equations. Nevertheless, a total of 
91 different formulations of the basic equations, in which mission 
performance level, mission performance quality, and training quality 
are related to training load, resource availability, and. external 
factors, have been estimated for the four aircraft maintenance units. 
These formulations all describe versions of .the general equations for 
which all estimated coefficients are statistically significant at, 
at least, the 10 percent level. The results of these empirical tests 
of the fundamental components of the conceptual model of OJT capacity 
are discussed in the following three subsections, 

5.3.1 Mission Performance Level 

Eight of the 91 empirical relationships estimated for the aircraft 
maintenance units have examined the association between the level of 
mission performance attained by a unit and the training load, resource 
availability, and external factors affecting the unit. As explained 
previously, only one usable indicator of the level of mission perfor- 
mance attributable to the aircraft maintenance units has been obtained 
in this study. The empirical results derived by using multiple linear 
regression analysis to develop estimates of the relationship between 
this indicator, FLYHRS , and the various indicators of training load, 
resource availability, and external factors are presented in Table 8. 

These results fail to reveal any consistent or strong association 
between the level of mission performance achieved by a unit and either 
the amount of training provided by the unit or — within the range of 
resource allotments observed in the study — the availability of 
resources to the unit. In only two of the estimated equations are 
training load variables related statistically significantly to FLYHRS, 
and in one of these instances, the sign of the estimated coefficient 
is contrary to prior expectations. Similarly, resource availability 
variables enter only three of the equations statistically significantly, 
and on two of these occasions the estimated coefficients have unantici- 
pated signs. This incidence and distribution of statistically signif- 
icant estimated coefficients can reasonably be expected to occur 
randomly in the absence of any actual association between the dependent 
and explanatory variables. Consequently, the empirical results provide 
no evidence of any interdependency between the level of mission per- 
formance, training load, and resource availability — at least in the 
low stress, peacetime situations observed in this study. 

5.3.2 Mission Performance Quality 

A total of 73 equations have been estimated for the aircraft main- 
tenance units to investigate the relationships between each of the 17 
indicators of mission performance quality and the various indicators 
of training loads, resource availability, and external factors. The 
detailed empirical results of these investigations are presented in 
Tables A-l through A-17 of the Appendix. 
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TABLE 8: Eapirical Results of Aircraft Maintenance Unit 
Regressions with FLYHRS as Dependent Variable 
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These results are generally supportive of the hypotheses embodied 
in the conceptual model of OJT capacity developed in Section 3.0. How- 
ever, this support is not unequivocal. No statistically significant 
coefficients have been estimated for any of the training load or 
resource availability variables in 15 of the 74 equations. Only the 
external factors variable, SEASON, has been determined to be a statis- 
tically significant explanatory variable in these relationships. 
Moreover, of the 82 statistically significant coefficients estimated 
for training load and resource availability variables in the remaining 
58 equations, 26 have signs opposite to prior expectations. 

Nevertheless, a substantial majority — 68 percent — of these 
estimated coefficients exhibit precisely the signs anticipated before 
the empirical analysis. In addition, the nunber of statistically 
significant coefficients estimated for the training load and resource 
availability variables Is considerably larger than the nunber that 
would be expected to originate through purely random processes. 

Furthermore, separate examination of just the coefficients esti- 
mated for the training load variables reveals that training load vari- 
ables enter 20 of the estimated equations statistically significantly 
— And that in 18 of these equations their estimated coefficients 
exhibit the anticipated signs. Thus, on only two occasions is the 
sign of an estimated coefficient of a training load variable inconsis- 
tent with prior expectations. These results are clearly indicative 
of a persistent and strong inverse relationship between the anount 
of training conducted in a' unit and the quality of mission performance 
attributable to the unit. 

Yet, these results also indicate that the preponderance of the 
unanticipated empirical results derived for the aircraft maintenance 
units must be associated with the resource availability variables. 
Nevertheless, even for these variables, the nunber of statistically 
significant estimated coefficients with signs consistent with prior 
expectations is substantially greater than the nunber of such estimates 
with unanticipated signs. A total of 62 statistically significant 
coefficients have been estimated for resource availability variables 
in 49 of the equations examining mission performance quality. Of 
these 62 coefficients, 38 have the anticipated signs; while only 24 
have signs contrary to expectations. With 63 percent of the estimated 
coefficients exhibiting the expected signs, the enpirlcal results are 
basically supportive of the presumed relationships between resource 
availability and the quality of mission performance. 

Finally, it is informative to examine separately the relationships 
estimated for each of the 17 indicators of mission performance quality. 
Thus, Table 9 presents, for each mission performance quality indicator, 
the incremental and cumulative frequency distributions of the signifi- 
cance levels of the regression coefficients estimated for the training 
load variables. Similar distributions for the estimated coefficients 
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TABLE 9: Distributions of Significance Levels of Training 
Load Coefficients in Mission Performance Quality 
Regressions for Aircraft Maintenance Units 
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of the resource availability variables appear in Table 10. Then, 
aggregate distributions pertaining to the coefficients estimated for 
both the training load and the resource availability variables are 
reported in Table 11. 

These tables clearly reveal that considerable differences exist 
among the frequency distributions associated with the various mission 
performance quality indicators. The signs of the coefficients esti- 
mated for some indicators are highly consistent with prior expectations 
for both the training load variables and the resource availability 
variables. For other indicators, such consistency is observed only 
for the training load variables or, in one case, only for the resource 
availability variables. For the remaining indicators, substantial 
inconsistency with prior expectations is revealed for both sets of 
explanatory variables. 

Some appreciation of the possible origins of these differences 
can be gained by considering in greater detail the interpretations of 
the various mission performance quality indicators. Each of the 17 
Lndicators captures some aspects of the quality of mission performance 
attributable to the aircraft maintenance units. Yet, there are clear 
distinctions among the perspectives on mission performance quality 
provided by the different indicators. 

Thus, the first six indicators — NGBORTS, NABORTS, NBORTS, 
NMNTC, NMATTR, and NFAIL — all represent measures of the absolute 
quality of primary mission performance achieved by the flying wings 
whose aircraft are serviced by the aircraft maintenance units. For 
these indicators, in general, the signs of the coefficients estimated 
for both the training load and the resource availability variables 
exhibit a high degree of consistency with prior expectations. Only 
the indicator NFAIL reveals any notable disagreement with anticipation, 
and even for this indicator, a majority of the estimated coefficients 
have the expected signs. 

The next five indicators — RR3HR, RR6HR, MEFFC, ONTIME, and 
SORTYS — appraise the ability of either the aircraft maintenance units 
themselves or else the flying wings whose aircraft are maintained on 
the units, to fulfill predetermined performance schedules or standards. 
The signs of the estimated training load coefficients associated with 
these indicators are uniformly identical to the predicted signs. How- 
ever, considerably less compatibility with prior expectations is 
observed for the coefficients of the resource availability variables. 
Only for RR3HR and RR6HR, the variables most clearly and directly 
related to quality of performance of maintenance activities, do a 
majority of the coefficients estimated for the resource availability 
variables have the antici >ated signs. 
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TABU 10: Distributions of Significance Levexe of Resource 
Availability Coefficients In Mission Performance 
Quality Regressions for Aircraft Maintenance Units 
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TABLE 11: Distributions of Significance Levels of Training Load and 
Resource Availability Coefficients in Mission Performance 
Quality Regressions for Aircraft Maintenance Units 
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The next four indicators of mission performance quality — NNMCU, 
NNMCUS, NFMCM, NPCNM — reflect discretionary, and hence to some extent 
arbitrary, classifications of the reasons for, and severity of, the 
lack of full mission capability of aircraft requiring maintenance. 
Although the signs of the training load coefficients derived for these 
indicators conform reasonably well with prior expectations, it is also 
true that the only two statistically significant estimated training 
load coefficients with unanticipated signs appear in equations pertain- 
ing to these indicators. Moreover, the resource availability coeffi- 
cients estimated for these indicators typically have signs opposite 
to those predicted. 

The remaining two mission performance quality indicators — NRCWU 
and BREPR — represent indirect, and somewhat conceptually questionable 
measures of the absolute quality of maintenance support provided by the 
aircraft maintenance units. Thus, the indicator NRCWU does not describ 
just the recurrence for some aircraft of precisely the same maintenance 
requirement recently treated by a unit. Rather, it also reflects the 
occurrence in close succession of unrelated maintenance requirements 
within the same aircraft component. Similarly, the indicator BREPR 
not only measures instances in which aircraft maintenance units are 
unable to accomplish repairs which are normally performed within the 
units, but also enumerates situations in which, as a matter of policy, 
units never perform certain repairs internally. Consequently, it is 
not surprising that, for these two indicators of mission performance 
quality, no statistically significant relationships are derived for 
the training load variables, and a substantial portion of the coeffi- 
cients estimated for the resource availability variables have unantic- 
ipated signs. 

Thus, in summary, the empirical results derived for the mission 
performance quality indicators reveal 

• A persistent and strong inverse relationship between 
the amount of training conducted within aircraft 
maintenance units and most indicators of mission 
performance quality. 

• A somewhat weaker, but demonstrable, association in 
the anticipated direction between the amount of 
resources available to the units and the quality of 
mission performance* 

In addition, these results are demonstrated most strongly for the 
mission performance quality indicators that directly measure the 
absolute quality of primary mission performance achieved by the flying 
wings whose aircraft are serviced by the aircraft maintenance units. 
The more equivocal results are associated with the more questionable 
indicators of mission performance quality. 
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5.3.3 Training Quality 



The final 10 equations estimated for the aircraft maintenance 
units examine the relationship between the quality of training con- 
ducted in a unit and the training load, resource availability, and 
external factors affecting the unit. The detailed empirical results 
derived in these examinations for the three indicators of training 
quality developed in this study are reported in Tables A-18 through 
A- 20 of the Appendix. In addition, Table 12 presents, for each 
training quality indicator, the incremental and cumulative frequency 
distributions of the significance levels of the regression coeffi- 
cients calculated for the training load variables alone, the resource 
availability variables alone, and both of these sets of variables in 
combination. 

Overall, these results reveal substantial deviations between the 
signs estimated for the coefficients of these variables and the signs 
predict for these coefficients. Only one-third of the estimated 
coefficients exhibit the anticipated signs. 

Yet, as with the indicators of mission performance quality, 
noticeable differences exist among the results derived for the dif- 
ferent training quality indicators. For QINSPC, the only statistic- 
ally significant training load coefficient has the expected sign, and 
the majority of the estimated resource availability coefficients 
exhibit signs consistent with prior expectations. Conversely, for 
PINSPC and SAFET, the signs of both coefficients calculated for train- 
ing load variables, and the signs of virtually all coefficients esti- 
mated for resource availability variables, are opposite to the 
predicted signs. 

While these results are unanticipated, a reasonable explanation 
for their occurrence can be readily deduced by examining the interpre- 
tations of the three training quality indicators in greater depth. 
All of the indicators are based on inspections of maintenance tasks 
performed not only by recent trainees, but also by experienced enlisted 
personnel. In addition, the indicators PINSPC and SAFET reflect the 
results of personnel inspections assessing the extent to which per- 
sonnel use approved procedures when performing maintenance tasks. 
Thus, the unexpected signs estimated for the training load and 
resource availability coefficients in the equations pertaining to 
PINSPC and SAFET quite possibly result from the application of uncon- 
ventional, unapproved, and more risky maintenance techniques by the 
more experienced, more fully trained personnel. 

The indicator QINSPC, in contrast, is based on the results of 
quality verification inspections evaluating "he quality of the repairs 
produced through the performance of maintenance tasks. To the extent 
that the unconventional maintenance techniques employed by experienced 
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personnel are quicker or more effective than the approved procedures, 
the estimated training load and resource availability coefficients will 
exhibit the predicted signs. Conversely, to the extent that the uncon- 
ventional maintenance procedures are less efficient than. the standard 
techniques, the signs of the estimated coefficients will be contrary 
to prior expectations. Therefore, since some of the modified mainte- 
nance procedures are likely to be more efficient, and others will 
probably be less efficient, than the standard techniques, the mixed 
results derived for QINSPC sees reasonable. 

Finally, it should be remembered that, because the available 
indicators of training quality all focus on post-training task per- 
formance, it is equally reasonable to interpret the indicators as 
measures of the quality of mission performance. This unavoidable 
ambiguity should be taken into account when assessing the empirical 
results as evidence of the appropriateness of the conceptual model 
of OJT capacity. 

5.3.4 Summary 

The empirical results derived by estimating the fundamental 
equations of the conceptual model of OJT capacity for four aircraft 
maintenance units indicate that 

• A consistently strong inverse relationship exists 
between the amount of training conducted by a unit 
and the quality of mission performance attributable 
to the unit. 

• A perceptible, but weaker, association in the antic- 
ipated direction prevails between the resources avail- 
able to a unit and the quality of the unit's mission 
performance. 

• The most robust correlations of training load and 
resource availability with the quality of mission 
performance are observed for the indicators of mission 
performance quality that directly measure the absolute 
quality of primary mission performance achieved by the 
flying wings whose aircraft are serviced by the air- 
craft maintenance units. 

• A possible association exists between the quality of 
training provided in a unit and the training load and, 
conceivably, the available resources of the unit. The 
most persuasive evidence supporting this relationship 
is obtained when training quality is measured on the 
basis of assessments of the results produced through 
maintenance activities. However, due largely to 




limited data availability and ambiguity in the inter- 
pretation of the obtainable training quality indicators, 
the existence of this correlation has not been defini- 
tively determined in this study. 

• At least in the relatively low stress, peacetime situ- 
ations examined in the study, no persistent inter- 
dependency is discernible among the level of mission 
performance achieved, the amount of training conducted, 
and the availability of resources in aircraft mainte- 
nance units. 

Thus, overall, the empirical analysis of the aircraft maintenance units 
does not provide unequivocal support for each of the fundamental rela- 
tionships embodied in the conceptual model of OJT capacity formulated in 
Section 3.0. Nevertheless, particularly when the limited availability 
of pertinent data and the ambiguous interpretation of obtainable data 
are acknowledged, the results of the analysis provide ample support for 
the basic principle underlying the model — the principle that the 
capacity of Air Force operational units to conduct OJT is not unlimited. 
Rather, it is ultimately determined by the mission responsibility and 
resource endowments of the units. 
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6.0 CONCLUSIONS AND RECOMMENDATIONS 



Despite severe limitations in the availability of pertinent data 
from existing sources, a persistent statistically significant relation- 
ship has been observed, for all Air Force operational units studied, 
between the quality of mission performance achieved by a unit and the 
amount of training conducted in the unit. However, the analytic results 
only moderately support the possibility of an association between the 
quality of training provided in a unit and the amount of training per- 
formed in the unit; and the evidence concerning the existence of inter- 
dependence between the level of mission performance attained by a unit 
and the training load of the unit is negligible and contradictory. 

Nevertheless, especially considering the relatively poor quality 
of the obtainable data and the absence of several important explanatory 
variables, the empirical results must be interpreted as extremely favor- 
able indications that an operational methodology for estimating the 
capacity of Air Force operational units to conduct OJT can ultimately 
be Implemented. Moreover, the results strongly attest that the con- 
ceptual model of OJT capacity formulated and preliminarily tested in 
this study can appropriately and effectively provide the fundamental 
premises and basic analytic structure for this methodology. 

However, the development of an implementable methodology will 
require further exploratory research, supported by a systematic pro- 
gram of data collection and retention designed to provide a compre- 
hensive set of indicators describing all of the variables contained 
in the conceptual model. The remainder of this section presents spe- 
cific recommendations concerning prospective data acquisition and 
research activities. 



6.1 Dafa Acquisition Recommendations 



The validation and calibration of a model of OJT capacity requires 
histor . al data. Specifically, for the model formulated in this report, 
data suitable for creating six different sets of indicators must be 
acquired. The types of indicators to be created include 

• Mission performance level indicators , describing the 
actions performed by Air Force operational units, and 
specifying the magnitudes of these actions. 

• Mission performance quality indicators , assessing the 
quality of the outputs directly or ultimately produced 
through the units' actions. Normally, these indicators 
will be based on standard quality control measures, 
such as tabulations of the number of errors committed 
or the speed of task performance. 
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• Training load Indicators , reporting the total quantity 
of training conducted within the units, including both 
currently recorded UGT enrollment and informal activ- 
ities such as qualification training. In addition, 
some assessment of trainees' progress in OJT would be 
desirable. 

• Training quality indicators , evaluating the quality of 
the training conducted in the units, preferably in terms 
directly relevant to the units' mission responsibilities. 

• Resource availability indicators , characterizing the 
resources allocated to the units for mission performance 
or training. The resource endowments described should 
include personnel, equipment, and financial assets. 
Moreover, if possible, measurements of personnel 
resources should be partitioned by skill. 

• External factors indicators , describing phenomena beyond 
control of the units that might affect their training or 
mission performance. Pertinent indicators for many units 
might include weather conditions, or the diversion of 
personnel to activities outside the units. 

The results derived in the current study clearly demonstrate that, 
for different types of operational units, the precise forms of the pro- 
spectively pertinent indicators for each of the sets listed above will 
be different. Therefore, to determine the particular data items that 
must be acquired to create such indicators, it will first be necessary 
to select the specific types of operational units for which the con- 
ceptual model of OJT capacity will be empirically analyzed in the next 
research effort. 

Then, it will be necessary to develop precise definitions of the 
indicators prospectively pertinent to each of the selected types of 
units. The definitions should reflect all aspects of each set of 
indicators believed to affect the units, but should be susceptible 
to as few extraneous influences or contradictory interpretations as 
possible. 

Next, the definitions should be decomposed to isolate the indi- 
vidual component data items from which the indicators are formed. 
Thus, for example, ratios should be split into their numerators and 
denominators, and measures of change over time should be expressed 
in terms of their underlying data series. 

Whenever possible, to minimize the expenditure of time and effort, 
the resultant desired data elements should then be obtained from estab- 
lished sources. In many instances, the desired data will correspond to 
information already routinely collected and maintained by the Air Force. 
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However, in other cases, the data, while regularly collected by 
the Air Force, may not be retained for a long enough period of time, 
or may not be preserved in a suitable form, to be readily usable for 
the proposed analysis. For example, some information may currently 
be compiled only for the squadron or the flying wing, while tabulations 
for individual work centers might be needed if the interactions among 
resources, training loads, and mission performance that, in combination, 
determine OJT capacity are to be thoroughly investigated. In these 
situations, revised data recording amd retention procedures should be 
established to assure the availability of the required information. 

Finally, some desired data elements may not presently be collected 
in any form by any identifiable agency. For these data elements, com- 
pletely new data acquisition and retention procedures must be estab- 
lished and implemented. 

However, for purposes of model validation and calibration, the 
data acquisition activities outlined previously need not be undertaken 
universally throughout the Air Force. Rather, a restricted number of 
operational units of each designated type can be selected to serve as 
representative prototypes for their general class of units. The estab- 
lished data collection, compilation, and retention procedures can then 
be applied exclusively to the selected units. Only after the OJT 
Capacity estimation methodology has been fully demonstrated, and the 
precise data items applicable to this methodology have been determined, 
should expanded data acquisition procedures be implemented in all oper- 
ational units. 



6.2 Data Analysis Recommendations 

The encouraging results derived in this study suggest that further 
research efforts to validate and calibrate the conceptual model of OJT 
capacity on the basis of improved data may ultimately prove to be highly 
beneficial to the Air Force. Specifically, using the information 
obtained through the data acquisition efforts outlined in the preceding 
section, empirical estimates of the coefficients of each of the model* s 
basic equations should be derived. Essentially, this involves the esti- 
mation of production functions describing the level and quality of 
mission performance and training provided in individual operational 
units. Other studies have attested to the difficulty of this task.^ 
However, the results produced in this study indicate that such relation- 
ships can successfully be established, at least for the factors most 
pertinent to estimating OJT capacity, and with sufficient accuracy to 
permit the derivation of satisfactory capacity estimates. 



See, for example, Z. Barzily, W. H. Marlow, and S. Zacks, "Survey 
of Approaches to Readiness, " Naval Research Logistics Quarterly , 
Arlington, Virginia, Office of Naval Research, March 1979, pp. 21-31. 
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The empirical analysis to be undertaken in the next research 
effort can be conducted in either of two ways. First, if the deriva- 
tion of improved empirical results is desired as quickly as possible, 
the equations in the conceptual model can be estimated using cross- 
sectional data describing conditions in a relatively large number of 
similar operational units during a relatively short period of time. 
However, this approach implicitly assumes that the functional rela- 
tionships determining OJT and mission performance are identical in 
all of these units, but typically does not provide sufficient data 
to test the validity of this assumption. 

Alternatively, the equations in the conceptual model can be esti- 
mated using time series data for a smaller number of operational units, 
but for a longer period of time. Obviously, adoption of this approach 
will necessarily delay the derivation of improved analytic results. 
However, it will also permit the direct comparison of the estimated 
equations derived for similar operational units to test the validity 
of the assumption that the corresponding functional relationships are 
identical. 

If this assumption is confirmed for any of the sets of operational 
units examined, a satisfactory OJT capacity estimation methodology could 
be directly established for these units on the basis of the estimated 
equations. Conversely, if the empirical results reveal differences 
among the estimated equations derived for presumably similar units, 
an investigation should be undertaken to determine the sources of the 
observed variability. In general, this would entail refinement of the 
conceptual model, possibly through the inclusion of additional explan- 
atory variables or through respecif ication of the presumed relationships 
among variables, to develop a formulation that accounts for the observed 
differences. 

Only if this effort is unsuccessful for some t ->e of operf.-tional 
units will implementation of an OJT capacity estimation methodology 
applicable throughout the entire Air Force entail separate estimation 
of the basic equations of the conceptual model of OJT capacity for each 
individual unit of that type. Moreover, ever ^ s task, while obvi- 
ously onerous and costly, is feasible and pott .tially beneficial. 

In particular, the derivation of satisfactory estimates of the 
fundamental equations of the conceptual model in the manner discussed 
above will permit the calculation of theoretically sound estimates of 
the capacity of operational units to conduct OJT. Development of such 
OJT capacity estimates will, in general, involve the application of 
mathematical optimization techniques such as linear programming to 
derive solutions to constrained maximization problems such as those 
specified, and illustrated, in Section 3.0. No calculations of this 
type have been performed in this study because of the severe data 
limitations confronted for the operational units examined. Any assess- 
ments of OJT capacity derived under these circumstances would have been 
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inconsequential and meaningless. However, improved data and refined 
estimates of the conceptual model of OJT capacity can reasonably be 
expected to result in an informative and useful analysis. 

Finally, the availability of acceptable estimates of the basic 
equations of the conceptual model of OJT capacity for a substantial 
number of operational units will permit the formulation and testing 
of procedures for estimating the OJT capacities of operational units 
at higher organizational levels by appropriately aggregating the 
capacities estimated for lower- level component units. At the extreme, 
this might entail the development of methodologies, applicable at the 
base or MAJCOM (Major Command) levels, that use decomposition algo- 
rithms to estimate the OJT capacities of the component units. More 
realistically, it should be possible to account for the interdepend- 
ences between higher level and lower level units by developing func- 
tional procedures that represent simple extensions of the OJT capacity 
estimation procedures applicable to the component units. For example, 
it may be possible to establish factors denoting base and MAJCOM 
capacity constraints that can be included directly in the OJT capacity 
estimation methodologies of the lower level units. In any event, the 
development of satisfactory aggregation procedures will permit the 
estimation of OJT capacities for virtually any organizational level 
within the .Air Force. 



63 



APPENDIX 



65 



ERIC 



E A-l 



: Empirical Results of Aircraft Maintenance Unit 
Regressions with NGBORTS as Dependent Variable 
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E A-2: finpirical Results of Aircraft Maintenance Unit 
Regressions with NABORTS as Dependent Variable 
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A-3: Empirical Results of Aircraft Maintenance Unit 
Regressions with NBORTS as Dependent Variables 
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lBLE A-4: Empirical Results of Aircraft Maintenance Unit 
Regressions with NMNTC as Dependent Variable 
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LE A-5: Empirical Results of Aircraft Maintenance Unit 
Regressions with NMATTR as Dependent Variable 
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'ABLE A-6: Empirical Results of Aircraft Maintenance Unit 
Regressions with NFAIL as Dependent Variable 
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Significantly different from 0 at the 5 percent level, 
significantly different from 0 at the 10 percent level. 
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UJLE A-7: Bnpirical Results of Aircraft Maintenance Unit 
Regressions with RR3HR as Dependent Variable 
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A-8: Bnpirical Results of Aircraft Maintenance Unit 
Regressions with RR6HR as Dependent Variable 
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9: Empirical Results of Aircraft Maintenance Unit 
Regressions with ONTIME as Dependent Variable 
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A-10: Empirical Results of Aircraft Maintenance Unit 
Regressions with SORTYS as Dependent Variable 
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J A-13: Empirical Results of Aircraft Maintenance Unit 
Regressions with NNMCUS as Dependent Variable 
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A-14: Bnpirical Results of Aircraft Maintenance Unit 
Regressions with NPMCM as Dependent Variable 
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TABLE A-17: Empirical Results of Aircraft Maintenance Unit 
Regressions with BREPR as Dependent Variable 
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TABLE A-1B: Empirical Results of Aircraft Maintenance Unit 
Regressions with QINSPC as Dependent Variable 




Significantly different frn 0 at the 5 percent level, 
0 "Significantly different from 0 at the 10 percent level, 
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TABLE A-19: Empirical Results of Aircraft Maintenance Unit 
Regressions with PINSPC as Dependent Variable 



Unit 


lemllion '*l!iclents (and ^statistics) 


R2 


P 


F-statistic 

fund AppTPtu 
of f fedora) 




TL _ 1 




E 


PTRN 


STRN 




IS KILL 


INEXP 


TRNRS 


SEASON 


1 


0,908 






'.652 
32) a 










0.230 


0.187 


5.37 
(l,18)» 


3 


0.801 










0.491 
(3.29) a 




-0.0190 
(-2.04) a 


0.453 


0.389 


7.04 


0.660 












0.443 
(2.93)a 


0.367 
(1.52)t> 


-0.0147 
(-1.52)b 


0.568 


0.476 


6.14 
(3,14) 3 


t 


0.894 








0.530 
15.69)* 










0.630 


0.611 


32.38 
(1,I9)» 


0.829 




0.297 
(1.97) fl 




0.629 
5,58) a 










0.745 


0.711 


21.95 
(2,15)* 



CD 

'significantly different fton 0 at the 5 percent level, 
Significantly different froa 0 at the 10 percent level. 
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TABLE A-20: Empirical Results of Aircraft Maintenance Unit 
Regressions with SAFET as Dependent Variable 



Unit 



Constant 



0.923 



TL 



PTRN 



Regression Coefficient! (and t-Btatlstlca) 



RE57 



0,209 
(Ul) a 



0,244 
(2,66)a 



LSK.ILL 



IMEXP 



Significantly different from 0 at the J percent level. 
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